NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 


THESIS 


DYNAMIC  BANDWIDTH  PROVISIONING  USING 
MARKOV  CHAIN  BASED  ON  RSVP 

by 

Yavuz  Sagir 
September  2013 

Thesis  Advisor:  Preetha  Thulasiraman 

Second  Reader:  Joshua  D.  Green 


Approved  for  public  release;  distribution  is  unlimited 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing 
instruction,  searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection 
of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  this  burden,  to  Washington  headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215 
Jefferson  Davis  Highway,  Suite  1204,  Arlington,  V A  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction 
Project  (0704-0188)  Washington  DC  20503. 


1 .  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

_ September  2013 _ Master’s  Thesis _ 


4.  TITLE  AND  SUBTITLE  5.  FUNDING  NUMBERS 

DYNAMIC  BANDWIDTH  PROVISIONING  USING  MARKOV  CHAIN  BASED 

ON  RSVP _ 

6.  AUTHOR(S)  Yavuz  Sagir 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Naval  Postgraduate  School 

Monterey,  CA  93943-5000 _ 


9.  SPONSORING  /MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


1 1 .  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the 
official  policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government.  IRB  Protocol  number _ N/A _ . 


12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited _ 


13.  ABSTRACT 

An  important  aspect  of  wireless  communication  is  efficiency.  Efficient  network  resource  management  and 
quality  of  service  (QoS)  are  parameters  that  need  to  be  achieved  especially  when  considering  network 
delays.  The  cooperative  nature  of  unmanned  ground  vehicle  (UGV)  networks  requires  that  bandwidth 
allocation  be  shared  fairly  between  individual  UGV  nodes,  depending  on  necessity.  In  this  thesis,  we  study 
the  problem  of  dynamic  bandwidth  provisioning  in  a  UGV  network.  Specifically,  we  integrate  the  use  of  a 
basic  statistical  model,  known  as  the  Markov  chain  with  a  widely  known,  network  bandwidth  reservation 
protocol,  known  as  the  Resource  Reservation  Protocol  (RSVP).  The  Markov  chain  results  are  used  with 
RSVP  to  identify  specific  bandwidth  allocation  requirements  along  a  path  such  that  data  transmission  along 
that  path  is  successful.  Using  a  wireless  simulation  program  known  as  Qualnet,  we  analyze  the  bandwidth 
efficiency  and  show  that  this  algorithm  provides  higher  bandwidth  guarantees  and  better  overall  QoS  when 
compared  with  solely  using  RSVP  in  wireless  communication  networks. 


14.  SUBJECT  TERMS  Wireless  Communication,  Resource  Reservation  Protocol,  Markov 
Chain  Process,  Dynamic  Bandwidth  Allocation 


17.  SECURITY 
CLASSIFICATION  OF 
REPORT 

Unclassified 


NSN  7540-01-280-5500 


18.  SECURITY 
CLASSIFICATION  OF  THIS 
PAGE 

Unclassified 


19.  SECURITY 
CLASSIFICATION  OF 
ABSTRACT 

Unclassified 


15.  NUMBER  OF 
PAGES 

95 


16.  PRICE  CODE 


20.  LIMITATION  OF 
ABSTRACT 


Standard  Form  298  (Rev.  2-89) 
Prescribed  by  ANSI  Std.  239-18 


I 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Approved  for  public  release;  distribution  is  unlimited 


DYNAMIC  BANDWIDTH  PROVISIONING  USING  MARKOV 
CHAIN  BASED  ON  RSVP 


Yavuz  Sagir 

Lieutanant  Junior  Grade,  Turkish  Navy 
B.S.S.E,  Turkish  Naval  Academy,  2006 


Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  ELECTRONIC  WARFARE  SYSTEMS 

ENGINEERING 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
September  2013 


Author:  Yavuz  Sagir 


Approved  by:  Preetha  Thulasiraman 

Thesis  Advisor 


Joshua  D.  Green 
Second  Reader 


Dan  C.  Boger 

Chair,  Department  of  Information  Sciences 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


An  important  aspect  of  wireless  communication  is  efficiency.  Efficient  network 
resource  management  and  quality  of  service  (QoS)  are  parameters  that  need  to 
be  achieved  especially  when  considering  network  delays.  The  cooperative  nature 
of  unmanned  ground  vehicle  (UGV)  networks  requires  that  bandwidth  allocation 
be  shared  fairly  between  individual  UGV  nodes,  depending  on  necessity.  In  this 
thesis,  we  study  the  problem  of  dynamic  bandwidth  provisioning  in  a  UGV 
network.  Specifically,  we  integrate  the  use  of  a  basic  statistical  model,  known  as 
the  Markov  chain  with  a  widely  known,  network  bandwidth  reservation  protocol, 
known  as  the  Resource  Reservation  Protocol  (RSVP).  The  Markov  chain  results 
are  used  with  RSVP  to  identify  specific  bandwidth  allocation  requirements  along 
a  path  such  that  data  transmission  along  that  path  is  successful.  Using  a  wireless 
simulation  program  known  as  Qualnet,  we  analyze  the  bandwidth  efficiency  and 
show  that  this  algorithm  provides  higher  bandwidth  guarantees  and  better  overall 
QoS  when  compared  with  solely  using  RSVP  in  wireless  communication 
networks. 
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I.  INTRODUCTION 


Communication  networks  have  become  an  integral  part  of  life  today. 
However,  one  of  the  main  bottlenecks  of  communication  networks  is  the 
availability  of  bandwidth.  This  is  especially  true  in  the  wireless  environment. 
Compared  with  wired/wireline  networks,  wireless  networks  offer  several 
advantages  over  wired  networks,  such  as  ease  of  mobility  and  speed  of 
deployment,  flexibility  and,  in  some  cases,  reduced  costs.  Wireless 
communication  has  become  a  pervasive  aspect  in  information  technology  as 
there  has  been  a  gradual  enhancement  in  quality  and  security  of  wireless 
communication  devices  together  with  a  decrease  in  related  costs.  Although 
wireless  communication  is  spreading,  resources  are  very  scarce.  While  more 
wired  network/bandwidth  is  created  when  new  physical  resources  are  added  to 
the  communication  network,  wireless  communication  requires  sharing  finite 
natural  resources,  specifically,  the  radio  frequency  spectrum  [1],  In  other  words 
wireless  devices  are  constrained  to  operate  in  a  certain  frequency  band.  This  is 
especially  true  in  the  use  of  wireless  technology  for  defense  operations. 

In  recent  years,  the  Department  of  Defense  (DoD)  has  dramatically 
increased  the  use  of  mobile  wireless  communication  devices  for  both  tactical  and 
non-tactical  applications.  One  area  in  which  wireless  communication  has  become 
an  important  tool  is  in  the  deployment  of  unmanned  ground  vehicles  (UGV).  A 
UGV  network  operates  as  an  intermittently  connected  system,  distributed  over 
large  geographic  areas.  They  function  as  a  cooperative  mobile  ad  hoc  network 
(MANET).  This  geographic  dispersion  forces  reliance  on  wireless  communication 
to  implement  the  control  systems  for  the  networks.  As  the  number  of  nodes  in  the 
network  grows,  the  bandwidth  required  to  continuously  update  all  nodes  may 
exceed  the  available  bandwidth.  This  forces  system  engineers  to  implement  a 
prioritization  scheme  to  determine  which  node  will  be  updated  during  each  time 
frame.  This  thesis  is  focused  on  developing  an  algorithm  for  dynamic  bandwidth 
allocation  for  mobile  UGV  nodes  using  basic  statistical  and  network  protocols. 
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A.  BASIC  CONCEPTS  OF  WIRELESS  COMMUNICATION 

To  facilitate  the  discussion  of  this  thesis,  this  chapter  provides  an 
introduction  to  the  basic  concepts  of  wireless  communication,  particularly  the 
physical  and  mathematical  meaning  of  bandwidth. 

To  start  the  discussion  on  wireless  communication  concepts,  it  is 
convenient  to  begin  by  understanding  the  main  parameter  of  network 
performance,  known  as  channel  capacity.  Channel  capacity  is  the  upper 
boundary  on  the  rate  of  information  that  can  be  transmitted  in  a  communication 
channel.  The  channel  capacity  that  a  radio  frequency  channel  can  support  is 
limited  by  Shannon’s  capacity  laws  [2], 

Bandwidth  is  generally  defined  as  being  a  measure  of  the  data  capacity  of 
a  link.  In  terms  of  wireless/digital  communication  or  signal  processing  the  word 
bandwidth  is  used  to  refer  to  the  analog  signal  bandwidth  measured  in  hertz.  In 
other  words  bandwidth  is  the  difference  between  the  upper  and  lower 
frequencies  in  a  continuous  set  of  frequencies.  According  to  Hartley’s  law,  the 
relation  between  bandwidth  and  data  rate  is  that  the  digital  data  rate  limit  (or 
channel  capacity)  of  a  physical  communication  link  is  proportional  to  its 
bandwidth  in  hertz  [3],  In  computer  networking,  bandwidth  is  a  measurement  of 
the  bit-rate  of  available  or  the  consumed  data  communication  resources 
expressed  in  bits  per  second.  Each  frequency  channel  has  an  associated 
bandwidth  which  is  simply  the  amount  of  frequency  space  in  the  band.  In  Figure 
1 ,  the  spectrum  of  bandwidth  for  various  frequencies  is  shown. 
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Figure  1 .  Frequency  spectrum  for  various  wireless  communication  channels 

from  [4], 

Channel  capacity  can  be  related  to  bandwidth  through  the  Shannon 
Hartley  theorem  given  in  Equation  (1 .1 ), 

C  =  B  log2(l  +  |;)  (0.1) 

where  C  is  the  channel  capacity  in  bits  per  second,  B  is  the  bandwidth  of  the 
channel  in  hertz,  S  is  the  average  received  signal  power  measured  in  watts,  N  is 
the  average  noise  or  interference  power  and  S/N  is  the  signal  to  noise  ratio.  The 
Shannon  formula  represents  the  theoretical  maximum  capacity  that  can  be 
achieved  [1].  It  must  be  noted  that  Eq.  1.1  is  based  on  the  assumption  that  the 
channel  is  noiseless. 

In  Equation  (1.2),  the  Shannon-Hartley  theorem  is  re-written  assuming 
that  the  channel  is  susceptible  to  Gaussian  Noise  (AWGN). 

C  =  W  log,  (1  h — ^— )  (0.2) 

N0W 

Here  the  average  received  power  is  denoted  as  P  measured  in  watts,  and 
the  noise  power  spectral  density  is  N0  measure  in  watts  per  hertz. 
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B.  PROBLEM  STATEMENT  AND  MOTIVATION 


An  important  aspect  of  wireless  communication  is  efficiency.  Efficient 
network  resource  management  and  quality  of  service  (QoS)  are  parameters  that 
need  to  be  achieved  especially  when  considering  network  delays.  The 
cooperative  nature  of  UGV  networks  requires  that  bandwidth  allocation  be 
shared  fairly  between  individual  UGV  nodes,  depending  on  necessity. 
Specifically,  nodes  that  require  more  bandwidth  for  data  transmission  should  not 
be  starved  and  arbitrarily  limited  by  bandwidth  resources  while,  conversely, 
nodes  requiring  less  bandwidth  should  not  be  over  allocated.  This  form  of 
bandwidth  management  is  known  as  statistical  multiplexing.  In  other  words,  the 
shared  transmission  links  adapt  instantaneously  to  the  traffic  demands  of  the 
node  connected  to  the  link.  This  dynamic  bandwidth  allocation  takes  advantage 
of  several  assumptions  of  cooperative  networks: 

•  All  users  are  typically  not  connected  to  the  network  at  one  time. 

•  Even  when  connected,  users  are  not  transmitting  data  (or  voice  or 
video)  at  all  times. 

•  Most  traffic  occurs  in  bursts  meaning  that  there  are  gaps  between 
the  transmission  of  groups  of  packets. 

While  bandwidth  allocation  schemes  for  wireless  networks  have  been 
studied  in  the  literature,  UGVs  would  benefit  from  implementation  of  a  dynamic 
bandwidth  provisioning  scheme.  The  successful  implementation  of  an  efficient 
bandwidth  provisioning  algorithm  for  UGV  networks  will  contribute  to  helping 
solve  the  limited  bandwidth  problem  and  prevent  failures  of  UGV  nodes  in  terms 
of  communication. 

In  this  thesis,  we  study  the  problem  of  dynamic  bandwidth  provisioning  in 
an  UGV  network.  Specifically,  we  integrate  the  use  of  a  basic  statistical  model 
known  as  the  Markov  chain  with  a  widely  known,  network  bandwidth  reservation 
protocol,  known  as  the  Resource  Reservation  Protocol  (RSVP). 
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The  Markov  chain  process  is  a  stochastic  process  which  can  be  used  in 
various  places  in  wireless  communication.  The  Markov  chain  process  is  useful 
for  the  following  tasks  : 

•  Modeling  wireless  channels, 

•  Estimating  future  traffic  demand  when  nodes  are  mobile, 

•  Dynamically  allocating  bandwidth  in  a  network  with  varying  QoS 
demands. 

RSVP  is  a  transport  layer  protocol  that  enables  Internet  applications  to 
obtain  differing  QoS  for  their  data  flows.  It  is  designed  to  meet  three  basic 
difficulties:  achieving  higher  bandwidth,  dealing  with  real  time  traffic,  and  dealing 
with  bursty  data.  The  goals  of  RSVP  can  be  summarized  as  follows  [5]: 

•  Accommodate  heterogeneous  receivers, 

•  Adapt  to  route  changes, 

•  Control  protocol  overhead, 

•  Use  network  resources  efficiently 

•  Accommodate  heterogeneous  underlying  technologies 

In  this  regard,  RSVP  is  an  ideal  candidate  for  facilitating  dynamic 
bandwidth  allocation  in  UGV  networks. 

Our  aim  in  briefly  introducing  the  RSVP  and  Markov  chain  models  above 
is  to  illustrate  the  motivation  of  using  these  techniques.  Particularly  we  wish  to 
articulate  the  basic  characteristics  of  the  models  and  how  they  are  amenable  for 
use  in  cooperative  UGV  networks.  Specifically,  the  RSVP  does  not  have  a 
mechanism  for  traffic  demand  prediction  and  the  Markov  chain  process  cannot 
be  used  as  a  network  tool  for  bandwidth  allocation.  Thus,  by  exploiting  the 
characteristics  of  both  and  integrating  them,  we  achieve  a  more  robust  and 
efficient  dynamic  bandwidth  allocation  algorithm. 

We  will  expand  upon  the  RSVP  and  Markov  chain  process  in  Chapter  III. 
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C.  THESIS  CONTRIBUTIONS 

The  contributions  of  the  research  proposed  in  this  thesis  lies  in  the 
creation  of  a  new  algorithm  that  dynamically  allocates  bandwidth  to  node  in  a 
UGV  network,  taking  into  consideration  the  channel  quality. 

This  thesis  makes  the  following  contributions: 

•  The  Markov  chain  process  is  implemented  in  a  UGV  network, 
where  each  UGV  node  has  some  velocity.  The  Markov  Chain 
model  is  demonstrated  to  predict  channel  conditions  at  each  UGV 
node.  The  result  of  the  Markov  chain  is  used  to  formulate  estimated 
bandwidth  requirements. 

•  The  RSVP  algorithm  is  implemented  and  integrated  with  the 
Markov  chain  process.  The  Markov  chain  results  are  used  as  input 
to  the  RSVP  process  such  that  the  RSVP  algorithm  can  either 
increase  or  decrease  the  bandwidth  requirement  at  each  node. 
More  specifically,  the  Markov  chain  results  are  used  with  the  RSVP 
to  identify  specific  bandwidth  allocation  requirements  along  a  path 
such  that  data  transmission  along  that  path  is  successful. 

•  Qualnet,  a  simulation  platform  for  the  wireless  environment  is  used 
to  simulate  the  algorithm  (integration  of  Markov  chain  processing 
with  RSVP).  We  analyze  the  bandwidth  efficiency  and  show  that 
this  algorithm  provides  higher  bandwidth  guarantees  and  better 
overall  QoS  when  compared  with  solely  using  RSVP  in  the  wireless 
communication  networks. 

D.  ORGANIZATION  OF  THE  THESIS: 

The  remainder  of  this  thesis  is  organized  as  follows.  A  comprehensive 
literature  review  on  various  aspects  of  static  and  dynamic  bandwidth  allocation  in 
wireless  networks  is  presented  in  Chapter  II.  In  Chapter  III,  the  general  Markov 
Chain  and  RSVP  algorithms  are  discussed.  The  integration  of  the  Markov  Chain 
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process  and  the  RSVP  algorithm  is  presented  in  Chapter  IV.  Chapter  V 
discusses  the  performance  evaluation  and  results  of  the  dynamic  bandwidth 
allocation  algorithm.  The  conclusions  and  directions  for  future  work  are  provided 
in  Chapter  VI.  The  appendix  contains  the  Qualnet  program  files  used  in  this 
research  for  simulations  along  with  supplementary  performance  results. 
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II.  RELATED  WORK 


In  terms  of  the  dynamic  bandwidth  allocation  problem  in  the  wireless 
communication  environment,  there  are  numerous  different  solutions  that  exist  in 
the  literature.  The  main  goal  of  these  techniques  is  to  increase  the  QoS  and 
bandwidth  efficiency. 

A.  DYNAMIC  BANWIDTH  PROVISIONING 

There  are  two  types  of  bandwidth  allocation  methods:  static  bandwidth 
allocation  and  dynamic  bandwidth  allocation.  Static  bandwidth  allocation  is 
generally  ineffective  in  terms  of  QoS.  This  is  because  a  channel  is  dedicated 
some  amount  of  predefined  bandwidth,  regardless  of  whether  it  uses  it  or  not, 
and  thus  the  bandwidth  resources  are  never  released  for  use  by  other  services. 
On  the  other  hand,  dynamic  bandwidth  allocation  works  better  in  environments  in 
which  bandwidth  is  in  high  demand. 

Dynamic  bandwidth  allocation  is  defined  in  the  literature  as  a  technique  by 
which  traffic  bandwidth  in  a  shared  telecommunications  medium  can  be  allocated 
on  demand  and  fairly  among  different  users  of  that  bandwidth.  This  is  a  form  of 
bandwidth  management,  and  is  essentially  the  same  thing  as  statistical 
multiplexing  [6]. 

As  part  of  the  bandwidth  management  process,  dynamic  bandwidth 
allocation  is  an  ongoing  process.  As  applications  are  engaged,  the  network 
allocates  a  portion  of  the  free  resources  to  each  of  the  applications,  carefully 
keeping  a  balance  that  ensures  each  application  has  sufficient  tools  to  function 
efficiently.  Once  a  particular  application  is  completed  and  is  no  longer  active,  that 
bandwidth  is  freed  up  and  available  for  use  by  other  applications  as  needed. 

One  of  the  chief  benefits  of  dynamic  bandwidth  allocation  is  that 

applications  which  may  require  considerable  resources  at  one  point  but  can 

function  with  much  less  at  a  later  time  are  automatically  adjusted  in  terms  of  the 

amount  of  bandwidth  set  aside  for  the  function.  In  the  interim,  any  bandwidth  that 
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remains  free  can  easily  be  allocated  to  other  resources.  This  is  different  from 
dedicating  bandwidth  to  specific  applications  since  that  bandwidth  is  not  available 
to  other  applications  even  when  it  is  not  actively  used. 

B.  MARKOV  CHAIN  PROCESS 

A  Markov  chain  is  a  discrete-valued  Markov  process.  Discrete-valued 
means  that  the  state  space  of  possible  values  of  the  Markov  chain  is  finite  or 
countable.  A  Markov  process  is  basically  a  stochastic  process  in  which  the  past 
history  of  the  process  is  irrelevant  if  the  current  state  is  known.  In  other  words,  all 
information  about  the  past  and  present  that  would  be  useful  in  saying  something 
about  the  future  is  contained  in  the  present  state. 

A  discrete-time  Markov  chain  is  one  in  which  the  system  evolves  through 
discrete  time  steps.  Thus,  changes  to  the  system  can  only  happen  at  one  of 
those  discrete  time  values.  In  a  UGV  network,  the  UGV  nodes  are  moving 
according  to  an  individual  mobility  pattern.  The  configuration  of  the  network  at  a 
time  t  can  be  determined  to  be  irrelevant  to  the  past  history  of  the  network.  The 
next  move  of  the  UGV  node  can  be  any  direction.  The  next  move  only  depends 
on  the  current  position. 

A  continuous-time  Markov  chain  is  one  in  which  changes  to  the  system 
can  happen  at  any  time  along  a  continuous  interval.  Similar  to  the  discrete 
Markov  chain,  the  past  history  is  irrelevant  to  the  future  state. 

The  Markov  chain  process  can  be  used  to  model  the  wireless  channel 
since  the  wireless  environment  fits  the  features  or  specialties  of  the  Markov  chain 
process.  In  the  literature,  discrete-time  Markov  chain  models  are  widely  used  to 
analyze  transmission  mechanisms  in  wireless  networks  since  techniques  for 
modeling  and  simulating  channel  conditions  play  an  essential  role  in 
understanding  network  protocol  and  application  behavior  [6].  The  wireless 
channel  can  be  modeled  using  the  Finite  State  Markov  Channel  (FMSC).  In 
general,  for  Markov  chains,  the  set  of  possible  values  for  each  state  is  a 
countable  set  S.  If  S  is  finite,  it  is  usually  taken  to  be  an  integer  number  as  S  = 
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{1 ,2,3, . . . ,M}  and  this  is  an  FMSC.  In  the  Markov  chain  process  if  the  state  space 
is  finite,  the  transition  probability  distribution  of  states  can  be  represented  by  a 
transition  matrix.  In  [6]  and  [7],  the  channel  is  modeled  using  a  discrete  time 
Markov  chain.  Similarly  in  [8]  channel  modeling  is  based  on  N-state  Markov 
chains  for  satellite  communication  (SATCOM)  systems  simulation.  The  main 
interest  of  this  modeling  is  to  simulate  the  wireless  environment  using  a  world¬ 
wide  radio-meteorological  databank. 

In  terms  of  bandwidth  allocation,  a  Markov  chain  based  model  for  dynamic 
bandwidth  allocation  in  a  Differentiated  Services  (DiffServ)  network  has  been 
proposed  [9].  DiffServ  is  a  computer  networking  architecture  that  specifies  a 
simple,  scalable  and  coarse-grained  mechanism  for  classifying  and  managing 
network  traffic  and  providing  QoS  on  different  services.  In  DiffServ  networks,  at  a 
timeslot,  the  proposed  Markov  chain  is  used  to  predict  the  bandwidth 
requirement  at  the  next  time  slot,  and  the  resource  (bandwidth)  is  then  allocated 
accordingly.  Such  a  pre-allocation  scheme  can  effectively  reduce  the  operation 
overhead  in  bandwidth  allocation  and  further  reduce  the  connection  blocking 
probability  in  DiffServ  networks. 

The  Markov  chain  process  can  also  be  used  to  estimate  the  future 
demand  in  a  cell  of  a  cellular  network  while  the  nodes  are  moving  to  dynamically 
allocate  bandwidth  and  QoS.  Location  and  request  prediction  have  been 
analyzed  in  [10].  The  authors  conclude  that  smart  wireless  networks  could 
deduce  future  client  locations  and  allocate  resources  in  advance  in  order  to 
mitigate  the  negative  effect  of  handovers  and  perform  paging  in  a  cost-effective 
manner.  Furthermore  in  [11],  which  also  studies  cellular  networks,  the  Markov 
chain  process  is  used  to  provide  dynamic  QoS  if  the  applications  are  capable  of 
adapting  to  the  level  of  QoS  provided  by  the  network,  which  may  vary  during  the 
course  of  a  connection.  Another  similar  approach  is  presented  in  [12],  The  main 
idea  of  [12]  is  the  utilization  of  the  bandwidth  pre-reservation  phase  in  the 
admission  control  protocol  through  a  Markovian  approach  in  order  to  predict  the 
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amount  of  bandwidth  needed  by  a  mobile  host  during  its  movements  among  the 
cells  it  will  probably  visit. 

In  wireless  personal  area  networks  such  as  wireless  body  area  sensor 
networks,  stations  or  devices  have  different  bandwidth  requirements  and  thus 
create  heterogeneous  networks.  The  idea  presented  in  [14]  represents  a  general 
discrete-time  Markov  chain  model  for  the  IEEE  802.15.4-based  networks.  It  takes 
into  account  the  slotted  carrier  sense  multiple  access  collision  avoidance 
(CSMA/CA),  which  is  a  scheme  to  control  access  to  the  transmission  medium 
[15],  and  guaranteed  time  slot  (GTS),  which  is  defined  as  a  mechanism  for 
serving  devices  that  required  dedicated  bandwidth  or  low  latency  transmission  in 
the  IEEE  802.15.4-based  networks.  It  considers  this  transmission  phenomena 
together  in  heterogeneous  a  traffic  scenario  and  non-saturated  condition.  For  this 
purpose,  the  standard  GTS  allocation  scheme  is  modified.  For  each  non-identical 
device,  the  Markov  model  is  solved  and  the  average  service  time  and  the  service 
utilization  factor  are  analyzed  in  the  non-saturation  mode. 

The  papers  discussed  above  ([6]  to  [14])  study  the  Markov  chain  process 
without  the  implementation  of  dynamic  bandwidth  allocation.  The  literature  that 
studies  channel  modeling  using  solely  the  Markov  chain  process  does  not  take 
into  consideration  the  benefit  of  implementing  specific  network  protocols  that  will 
aid  in  dynamic  bandwidth  provisioning.  In  other  words,  the  networking 
perspective  that  comes  from  using  an  RSVP  type  protocol  has  not  been 
considered  in  the  literature.  Thus,  the  approaches  studied  are  purely  from  a 
statistical  point  of  view. 

C.  RESOURCE  RESERVATION  PROTOCOL 

The  RSVP  is  a  transport  layer  protocol  that  is  designed  to  reserve 
resources  across  a  network.  RSVP  is  based  on  signaling  messages  that  traverse 
the  network,  allocating  resources  along  the  way.  RSVP  requests  resources  for 
simplex  (unidirectional  communication)  traffic  flows  (i.e,  a  traffic  stream  that  flows 
in  only  one  direction  from  the  sender  to  one  or  more  receivers.)  Thus,  a 
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bidirectional  exchange  of  data  between  a  pair  of  nodes  actually  constitutes  two 
separate  RSVP  simplex  sessions.  RSVP  is  receiver-initiated,  because  sender 
initiation  does  not  scale  well  to  large  scenarios  in  which  there  are  heterogeneous 
receivers.  There  are  two  types  of  primary  messages  in  RSVP.  They  are  path 
messages  and  reservation  messages.  In  multicast  scenarios,  the  devices  send 
out  only  one  PATH  message  to  multiple  receiving  devices,  thus  conserving 
network  bandwidth  [16].  A  path  message  is  sent  to  indicate  to  the  receiver  that 
the  sender  is  requesting  an  RSVP  capable  path.  Since  the  RSVP  is  end  to  end, 
the  path  from  sender  to  receiver  must  be  determined  and  agreed  upon.  In  other 
words  the  resources  required  along  every  hop  of  that  path  must  satisfy  the 
requirements  of  the  sender.  Once  the  PATH  message  is  received  by  the 
receiver,  it  sends  back  towards  the  sender  a  RESV  message.  The  receiver  will 
use  the  RESV  message  to  request  the  specific  reservation  parameters  along  the 
path.  This  is  known  as  QoS  reservation.  Figure  2  and  3  illustrate  the  mechanism 
of  RSVP  PATH  and  RESV  messages  respectively. 


Figure  2.  Illustration  of  the  flow  of  PATH  messages  from  sender  (SI )  to  three 
different  receivers  (Ml  ,M2,  and  M3)  via  multiple  routers  (R1 ,  R2,  and 

R3)  from  [17]. 


13 


Ml 


J 

r 

\ 


n  m2 


Figure  3.  Illustration  of  the  flow  of  RESV  messages  from  receivers  to  sender 

once  PATH  messages  are  processed  from  [17]. 

It  must  be  noted  that  RSVP  is  not  a  routing  protocol  but  works  well  with 
current  routing  protocols.  To  obtain  the  routes,  an  RSVP  process  consults  the 
local  routing  databases.  Routing  protocols  determine  where  packets  get 
forwarded;  RSVP  is  only  concerned  with  the  QoS  of  those  packets  that  are 
forwarded  in  accordance  with  routing  [16]. 

RSVP  is  a  soft-state  protocol,  meaning  that  the  reservation  must  be 

periodically  refreshed  or  it  expires.  This  supports  dynamic  automatic  adaptation 

to  network  changes  in  order  to  efficiently  accommodate  large  groups  of  nodes 

since  the  membership  of  a  group  and  the  resulting  route  path  are  likely  to  change 

with  time.  The  RSVP  protocol  specifies  the  creation  of  “soft  states”  that  can  be 

built  and  destroyed  incrementally  in  the  routers  and  the  hosts.  The  reservation 

information,  or  state,  is  cached  in  each  hop  tasked  with  managing  resources.  If 

the  network's  routing  protocol  alters  the  data  path,  RSVP  attempts  to  reinstall  the 

reservation  state  along  the  new  route.  When  refresh  messages  are  not  received, 

reservations  time  out  and  are  dropped,  releasing  bandwidth.  The  sender 

refreshes  PATH  messages,  and  the  receiver  refreshes  RESV  messages. 

Because  RSVP  sends  its  messages  as  best-effort  IP  datagrams  with  no 

reliability,  some  messages  might  be  lost,  but  the  periodic  transmission  of  refresh 

messages  by  hosts  and  routers  compensates  for  the  occasional  loss  of  an  RSVP 
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message.  To  ensure  receipt  of  refresh  messages,  the  network  traffic  control 
mechanism  must  be  statically  configured  to  grant  some  minimal  bandwidth  for 
RSVP  messages  to  protect  them  from  congestion  losses.  At  any  time,  the 
sender,  receiver  or  other  network  device  providing  QoS  can  terminate  the 
session  by  sending  a  PATH-TEAR  or  RESV-TEAR  message  [16]. 

It  is  possible  that  devices  along  an  RSVP  path  may  reject  resource 
requests.  If  the  reservation  is  rejected  due  to  lack  of  resources,  the  requested 
application  is  immediately  informed  that  the  network  cannot  currently  support  that 
amount  and  type  of  bandwidth  or  the  requested  service  level.  The  application 
determines  whether  to  wait  and  repeat  the  request  later  or  to  send  the  data 
immediately  using  best-effort  delivery.  QoS-aware  applications,  generally  begin 
sending  new  RSVP  request  messages  immediately  on  a  best-effort  basis,  which 
is  then  upgraded  to  QoS  when  the  reservation  is  accepted. 

RSVP  has  been  upgraded  and  proposed  in  a  new  version  called  Resource 
Reservation  Protocol-Traffic  Engineering  (RSVP-TE).  It  is  an  extension  of  the 
RSVP.  RSVP-TE  still  supports  the  reservation  of  resources  across  an  IP 
network.  The  differences  lie  in  the  ability  of  RSVP-TE  to  allow  the  use  of  Multi¬ 
protocol  Label  Switching  (MPLS).  MPLS  is  a  networking  standard  designed  to 
speed  up  network  traffic  flow.  MPLS  involves  setting  up  a  specific  path  for  a 
given  sequence  of  packets,  identified  by  a  label  put  in  each  packet,  thus  saving 
the  time  needed  for  a  router  to  look  up  the  address  of  the  next  node  for 
forwarding.  MPLS  is  called  multiprotocol  because  it  works  with  various  network 
protocols.  For  this  reason,  MPLS  can  carry  more  and  different  mixtures  of  traffic. 
Using  MPLS  with  RSVP  provides  a  more  robust  resource  allocation  mechanism. 
In  this  thesis,  the  RSVP-TE  protocol  is  used  for  our  study.  It  is  also  the  primary 
protocol  used  in  Qualnet,  the  simulation  platform  used. 
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III.  GENERAL  DISCUSSION  OF  MARKOV  CHAIN  ALGORITHM 

AND  RSVP 


In  this  chapter  we  first  give  a  general  idea  of  the  Discrete  Time  Markov 
Chain  process  used  in  the  researched  presented  in  this  thesis.  We  then  explain 
some  of  the  aspects  related  to  our  Markov  Chain  model.  We  also  present  a  more 
detailed  view  of  how  RSVP  works  in  communication  networks. 

A.  GENERAL  VIEW  OF  MARKOV  CHAIN 

As  was  discussed  in  Chapter  II,  the  Markov  chain  is  a  stochastic  process. 
For  such  a  process,  the  knowledge  of  the  outcome  of  any  preceding  node 
movement  does  not  affect  our  predictions  for  the  next  movement.  In  this  process 
a  set  is  defined  as  {X(t),  t  =  0,1,2,....}  whose  number  of  elements  is  finite  and 
denoted  with  real  positive  numbers.  Thus,  X(t)  =  i  represents  that  the  process  is 
in  state  i  at  an  instant  of  time  t  and  takes  discrete  values.  It  is  said  that  there  is  a 
fixed  probability,  py  that  indicates  the  chain  going  from  a  state  i  to  a  state  j  in  the 
next  time  t.  This  is  known  as  the  Markov  property  [1 8]. 

Equation  (3.1)  is  used  to  illustrate  the  memoryless  property  of  the  Markov 
chain  process.  This  property  indicates  that  given  the  present  state,  the  next  state 
is  conditionally  independent  of  the  past.  For  a  Markov  chain,  it  can  be  visualized 
as  a  process  moving  from  one  state  to  another  state  [19].  The  transitioning 
between  states  will  be  discussed  in  Section  NIB. 

P[X(tk)  =  Xk+l  I  X(tk  =  xk, . ,*(*,)]  =  P[X(tk)  =  XM  I  X(tk  =  x,)]  (0.3) 

where: 

X(tk),  Current  Sample 

X(tk+ j)  Future  Sample 

X^), . Past  Samples 

xk  State  of  sample  in  the  moment  k. 
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There  are  three  important  elements  to  Markov  Chains; 

•  Probability  Transition  Matrix  P, 

•  Transition  Diagram 

•  Steady  State  Vector  tt 

1.  Probability  Transition  Matrix 

The  switch  between  states  is  established  in  the  probability  transition 
matrix  P  (in  the  literature  it  is  also  described  by  T).  Each  element  of  the  matrix 
represents  the  probability  that  the  node  switches  to  another  state  or  remains  in 
the  current  state.  These  switches  are  called  transitions.  P  is  a  square  matrix 
whose  order  is  the  same  as  the  number  of  states. 

Each  element  of  P  must  satisfy  the  following  two  conditions: 

PtJ>0  i,j  =  0,1 ,2,3....  (0.4) 

where 

PUj  =  probability  that  the  current  node  is  in  state  i  given,  it  was  in  state  j  at  a  time 
immediately  preceding  the  current  time, 
i  =  transition  state  i. 

j  =  transition  state  j. 

In  addition,  it  is  also  important  that  the  sum  of  each  row  must  be  equal  to 
one  in  the  transition  matrix.. 

ZiP  =  1,  i  =  1,2,3, .  (0.5) 

The  structure  of  a  probability  transition  matrix  is  shown  in  Figure  3. 
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Figure  4.  Transition  matrix  for  Markov  chain  with  “n”  states. 

2.  Transition  Diagram: 

A  Markov  chain  transition  matrix  can  be  also  represented  graphically 
where  each  node  represents  a  state  of  the  system  and  is  numbered  accordingly, 
and  a  directed  arc  connects  state  i  to  state  j  if  a  one-step  transition  from  i  to  j  is 
possible.  Figure  4  illustrates  a  typical  transition  diagram  for  a  system  with  three 
states.  The  corresponding  transition  matrix  is  also  given. 


Figure  5.  Probability  transition  diagram  for  3  state  Markov  chain. 
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3.  Steady  State  Vector: 

One  of  the  important  elements  in  the  Markov  Chain  process  is  the  steady 
state  vector  tt,  which  represents  the  total  appearing  percentage  of  a  state  in  a 
Markov  Chain.  If  a  system  is  in  steady  state,  then  the  recently  observed  behavior 
of  the  system  will  continue  into  the  future.  In  stochastic  systems,  the  probabilities 
that  various  states  will  be  repeated  will  remain  constant. 

Given  a  square  matrix  A,  we  say  that  the  number  ^  is  an  eigenvalue  of  A 
if  there  exists  a  nonzero  vector  X  satisfying  AX=^X  .  In  this  case,  we  say  that  X 
is  an  eigenvector  of  A  corresponding  to  the  eigenvalue  ^ .  In  terms  of  the  Markov 
chain  process,  a  steady-state  vector  of  a  regular  Markov  chain  is  an  eigenvector 
for  the  transition  matrix  corresponding  to  the  eigenvalue  1 . 

The  steady  state  vector  can  be  computed  as 

PK  (0.6) 

where 

P  Probability  transition  matrix 

n  steady  state  probability  vector 

1  column  vector  of  ones  1T  =  (1 ,1 , . ) 

The  sum  of  its  elements  must  be  equal  to  one  as  is  shown  in  Eq.  3.5 

£;tt  =  1  (0.7) 

The  more  intricate  properties  of  the  Markov  Chain,  such  as  reducibility, 
peridocity,  recurrence,  ergodicity,  have  been  thoroughly  explained  in  the 
literature  [19], [20], [21], [22],  While  these  properties  are  beyond  the  scope  of  this 
thesis,  it  must  be  noted  that  they  are  important  for  understanding  the  dynamics  of 
the  Markov  chain  process. 
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B.  MARKOV  CHAIN  CHANNEL  MODELING 


In  a  typical  wireless  communication  system,  the  channel  conditions  vary 
with  time.  The  transmitted  signals  are  perturbed  by  additive  thermal  noise, 
frequency  and  time  selective  fading  and  interference  from  other  transmitters. 
Relative  mobility  between  transmitters  and  receivers  causes  the  channel 
conditions  to  change  accordingly. 

In  communication  systems,  the  Markov  chain  is  used  to  characterize  the 
wireless  channel  because  it  represents  some  of  the  phenomena  that  affect  the 
signal  during  its  wireless  transmission.  Thus,  by  using  a  Markov  chain,  we  are 
able  to  model  a  real  wireless  channel. 

A  Markov  chain  is  said  to  be  time  homogeneous  if 

P(X,)  =  i\P(X,_,)=  j  is  independent  of  time  index,  lim  p(k )  =  n 

If  the  Markov  chain  is  time-homogeneous,  then  the  transition  matrix  P  is 
the  same  after  each  step,  so  the  k-step  transition  probability  can  be  computed  as 
the  k-th  power  of  the  transition  matrix,  Pk. 

The  following  questions  are  important  in  the  study  of  the  Markov  Chain; 

a.  Does  a  n  so  that  n  =  nP  ?  If  such  a  ^exists,  it  is  called  a 
stationary  distribution. 

b.  If  a  unique  stationary  distribution  exists,  does  lim p{k)=K  for 

k  -»co 

all  P[0].  In  other  words  does  the  distribution  of  the  Markov  chain  converge  to  the 
stationary  distribution 

P[X(tk )  =  Xt+l  I  X(tk  =  xk, . ,*(*,)]  =  P[X(tk)  =  Xk+]  I  X(tk  =  x,)] 

Since  the  constant  Markov  process  has  the  property  of  stationary 
transitions,  the  transition  probability  is  independent  of  the  time  index  t  and  can  be 
written  as: 


t  =  P(X(n  +  1)  =  x(k)|  X(n)  =  x(j)  )  (0.8) 
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for  all  t=0,1,2,...  and  i,je{0,1 ,2,...,K-1}. 

The  model  is  so  characterized  by  P  composed  by  the  /^elements,  tt  for 

which: 


n'P  =  n'  (0.9) 

where  n  is  the  transpose  of  tt. 

In  communication  networks,  the  Gilbert-Eliott  model  is  a  wireless  channel 
model  based  on  a  two-state  (bad  and  good)  Markov  chain  which  is  widely  used 
for  describing  burst  error  patterns  in  transmission  channels.  In  other  words,  the 
Gilbert-Eliott  channel  is  a  varying  binary  symmetric  channel  with  crossover 
probabilities  determined  by  a  binary-state  Markov  chain  process.  In  a  binary 
symmetric  channel,  it  is  assumed  that  a  bit  is  usually  transmitted  correctly,  but 
rarely  will  it  be  flipped  with  a  small  probability  called  the  crossover  probability. 
This  crossover  probability  is  denoted  as  e.  The  cross  over  probability  is  strictly 
related  to  physical  parameters  (such  as  modulation  scheme,  carrier  frequency 
and  instantaneous  user  speed)[7]. 

As  mentioned  in  Chapter  II,  the  wireless  channel  can  be  described  with  a 
FMSC,  completely  characterized  by  its  state  transition  matrix  P,  steady  state 
vector  tt,  and  crossover  probability  vector  e  (fixed  error).  The  tt  and  e  (average 
error  probability  which  can  be  defined  as  a  probability  that  a  transmitted  signal  is 
received  in  error)  do  not  vary  generally  during  the  connection  because  they  are 
independent  from  transmission  rate  and  host  speed.  Matrix  P,  instead,  has  the 
dynamic  elements,  because  they  depend  on  physical  parameters  such  as  the 
wavelength  of  the  carrier  signal  and  transmission  rate  [7][12], 
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C.  GENERAL  VIEW  OF  RSVP 

In  this  section  we  give  a  general  idea  of  how  RSVP  works  and  then  focus 
on  some  important  aspects  of  RSVP,  such  as  refreshment  signaling  or  time  and 
bandwidth  reduction. 

In  RSVP,  a  data  flow  is  characterized  as  a  sequence  of  datagrams  that 
have  the  same  source,  destination  (this  could  be  more  than  one  machine)  and 
QoS.  A  basic  RSVP  reservation  request  consists  of  two  key  concepts:  flowspec 
and  filterspec.  This  pair  is  also  called  the  flow  descriptor.  RSVP  reserves 
resources  for  a  flow  which  is  identified  by  the  destination  address,  the  protocol 
identifier  (TCP  or  UDP)  and  the  destination  port.  The  flowspec  specifies  the 
desired  QoS  parameters  and  is  used  by  the  receiver  in  RESV  messages.  The 
flowspec  in  a  reservation  request  will  generally  include  a  service  class  and  two 
sets  of  numeric  parameters,  Rspec  and  Tspec.  Rspec  defines  the  desired  QoS, 
and  Tspec  describes  the  data  flow  [1 6]. 

On  the  other  hand,  the  filterspec  defines  the  set  of  data  packets  (the  flow) 
to  receive  the  QoS  defined  by  the  flowspec.  The  flowspec  is  used  to  set 
parameters  in  the  node’s  packet  scheduler,  which  is  a  link  layer  dependent 
mechanism  used  to  determine  which  particular  packets  are  forwarded  and  which 
achieves  the  desired  QoS  for  each  outgoing  interface,  while  the  filterspec  is  used 
to  set  parameters  in  the  packet  classifier,  which  determines  the  QoS. 

During  reservation  setup,  the  decision  to  follow  through  on  an  RSVP  QoS 
request  is  passed  to  two  control  mechanisms:  admission  control  and  policy 
control.  Admission  control  determines  whether  the  node  has  sufficient  available 
resources  to  supply  the  requested  QoS.  Before  a  reservation  can  be  established, 
the  RSVP  process  must  also  consult  the  policy  control  to  ensure  that  the 
reservation  is  administratively  permissible.  If  both  controls  succeed,  parameters 
set  in  the  packet  scheduler  are  executed;  otherwise,  the  RSVP  returns  an  error 
notification. 
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In  RSVP,  to  make  a  reservation,  RSVP  messages  carrying  reservation 
requests  originate  at  receivers  and  are  passed  upstream  towards  the  sender 
since  RSVP  is  receiver-oriented  as  mentioned  in  Chapter  II.  At  each  intermediate 
node,  two  actions  occur: 

•  The  RSVP  passes  the  request  to  admission  and  policy  control  and 
the  request  is  executed 

•  A  reservation  request  is  propagated  upstream  towards  the 
appropriate  senders.  The  set  of  sender  hosts  to  which  a  reservation 
request  is  propagated  is  called  the  scope  of  that  request. 

The  basic  reservation  model  is  one-pass.  In  this  model  a  receiver  sends  a 
request  upstream,  and  at  each  node  in  the  path  the  request  either  is  accepted  or 
rejected.  This  does  not  provide  a  way  to  determine  an  end-to-end  service. 
Therefore  another  model  is  used,  known  as  One-Pass  with  Advertising  (OPWA). 
With  this  scheme,  RSVP  control  packets  are  sent  downstream,  allowing  the  flow 
of  data  paths  to  gather  information  that  can  be  used  to  predict  the  end-to-end 
QoS. 


Much  of  the  RSVP’s  flexibility  resides  in  the  vector  reservation  attributes 
called  the  styles.  The  attributes  including  the  following: 

•  Sharing  attributes:  There  are  two  types  of  shared  attributes, 
known  as  shared  and  distinct.  Shared  attributes  require  that  all 
senders  share  the  same  reservation  whereas  distinct  attributes 
allows  receivers  to  support  a  distinct  reservation  for  each  sender’s 
traffic. 

•  Sender  selection  attributes:  In  general,  the  sender  selection 
attributes  control  how  senders  are  selected.  There  are  three  types 
of  sender  selection  attributes  known  as  explicit,  wildcard  and 
assured.  The  explicit  attribute  is  used  when  the  RESV  filterspec 
message  dictates  the  explicit  selection  of  senders  that  will  receive 
the  reservation.  With  the  wildcard  attribute,  RESV  has  no 
filterspecs.  In  other  words,  all  upstream  senders  can  be  selected  for 
the  reservation.  The  assured  attribute  is  a  hybrid  of  the  wildcard 
and  explicit.  Like  the  explicit  attribute,  the  filterspecs  explicitly  select 
those  senders  that  will  receive  reservation  packets,  but  like  the 
wildcard,  the  reservations  are  placed  towards  all  senders, 
regardless  of  which  senders  have  been  selected. 
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As  explained  in  Chapter  II,  there  are  two  types  of  primary  messages  in 
RSVP,  PATH  messages  and  RESV  messages.  These  messages  are  used  to 
allocate  enough  bandwidth  and  desired  QoS  between  receiver  and  sender  after 
using  the  reservation  model  and  determining  reservation  styles. 

Some  aspects  of  RSVP  which  we  focus  on  in  this  thesis  are  the 
refreshment  signaling,  time  and  RSVP  bandwidth  reduction.  RSVP  is  a  soft-state 
protocol,  which  means  that  the  state  times  out  if  it  is  not  refreshed  soon  enough. 
RSVP  maintains  state  in  each  node.  States  of  a  node  need  to  be  periodically 
refreshed,  thus  Refresh  Messages  (actually  they  are  just  PATH  and  RESV 
messages)  are  required.  These  refreshed  messages  are  used  for 

•  State  synchronization  between  neighbor  nodes  and 

•  Recovery  from  lost  RSVP  messages. 

Periodic  transmission  of  refresh  messages  by  nodes  (host  and  routers)  is 
expected  to  handle  the  occasional  loss  of  an  RSVP  message.  Suppose  that  a 
node  generates  a  RESV  refresh  every  R  seconds.  The  node  must  randomize  the 
inter-transmission  interval,  in  the  range  [0.5R,  1.5R]  [16], [23].  Note  that  the  value 
of  R  is  carried  in  each  refresh  messages.  Suppose  that  the  cleanup  timeout  is  set 
to  K  times  the  refresh  timeout  period,  and  TL  is  the  overall  lifetime  of  state  before 
it  is  deleted  [23],  The  equation  of  lifetime  of  reservation  is; 

TL=(K  +  0.5)*l.5*R  (0.10) 

If  K  is  an  integer  and  R  is  a  constant,  then  RSVP  can  tolerate  K-1  successive 
RSVP  packet  losses  without  falsely  deleting  state  [16]. 

In  [16],  the  default  K  value  is  3  and  the  refresh  time  is  30  sec.  When  R  is 
changed  dynamically  or  if  the  sender  increases  R  rapidly  but  one  or  more 
successive  packets  is  lost,  the  receiver’s  node’s  value  of  R  may  be  out  of  date.  In 
[16],  this  problem  is  solved  by  limiting  the  rate  of  increase  in  R.  The  limit  on  how 
fast  it  may  increase  depends  on  the  maximum  slew  ratio,  which  is  a  fixed 
constant  that  the  ratio  of  R  in  two  successive  timeout  intervals  cannot  exceed. 
Taking  into  consideration  that  the  maximum  slew  ratio  is  0.3  and  the  default 
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value  of  K  is  3  as  mentioned  above,  one  packet  may  be  lost  without  state  timeout 
while  R  is  increasing  30  percent  per  refresh  cycle. 

Another  important  aspect  of  RSVP  is  bandwidth  reduction  instead  of 
tearing  down  the  reservation  as  proposed  in  [24],  The  bandwidth  allocated  to  an 
individual  reservation  may  be  reduced  due  to  a  variety  of  reasons,  such  as 
preemption.  In  some  cases,  when  some  of  the  reservation  bandwidth  is  needed 
for  other  purposes,  instead  of  tearing  down  the  reservation,  the  endpoints  can 
negotiate  a  new  (lower)  bandwidth.  As  given  in  an  example  in  [24],  two 
aggregate  flows  with  differing  priority  levels  may  traverse  the  same  router.  If  that 
router  reaches  bandwidth  capacity,  the  router  then  has  two  choices:  deny  the 
request  or  preempt  an  existing  lower  priority  reservation  to  make  room  for  the 
new  or  expanded  reservation.  If  the  flow  is  preempted,  the  RSVP  clearly  does 
not  terminate  all  the  individual  flows  if  an  aggregate  is  torn  down  [25].  On  the 
other  hand  [24]  describes  a  method  where  only  the  minimum  bandwidth  is  taken 
away  from  the  lower  priority  aggregated  reservation  and  the  entire  reservation  is 
not  preempted.  A  similar  approach  can  be  modified  for  our  problem. 

When  a  reservation  partially  fails,  a  Reservation  Error  (ResvErr)  message 
is  generated.  A  new  error  subcode  (the  ERROR  SPEC)  is  required  and  it  is 
added  to  the  ResvErr  message.  This  indicates  the  flowspec  that  is  reserved.  The 
bandwidth  indication  in  this  flowspec  should  be  less  than  the  original  reservation 
request. 
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IV.  INTEGRATION  OF  MARKOV  CHAIN  WITH  RSVP 


In  this  chapter  we  first  explain  the  description  of  our  proposed  algorithm 
and  then  discuss  our  approach  in  terms  of  integrating  the  Markov  chain  with 
RSVP. 

A.  MODEL  DESCRIPTION 

In  our  model,  the  transition  matrix  shows  the  probabilities  of  changing  the 
states  of  the  link.  The  initial  vector  shows  (at  time  t  =0)  the  state  between  the 
nodes.  Some  aspects  of  our  model  are  discussed  in  the  following  sections. 

1.  Parameter  Used  to  Model  Markov  Chain  Channel 

Building  a  transition  matrix  and  the  initial  state  probability  vector  is  the  first 
step  of  the  Markov  chain  process.  It  is  essential  that  the  parameter  or  term  which 
is  chosen  to  exhibit  the  state  transitions  behave  according  to  the  Markov  chain 
process.  In  other  words  it  has  to  have  all  the  necessary  properties  such  as  being 
random,  memoryless,  etc. 

In  the  literature  of  wireless  communication  and  channel  modeling,  the 
attenuation,  the  SNR  value,  and  transmission  rates  are  chosen  to  build  the 
transition  matrix  [4],  [5].  In  our  model  we  prefer  to  choose  average  transmitted 
power  and  average  pathloss,  which  are  similar  to  SNR  and  attenuation. 

2.  Building  Initial  State  (Probability  Vector) 

One  of  the  main  steps  of  the  Markov  chain  process  is  getting  an  initial 
state  probability  vector  since  the  initial  state  vector  and  transition  matrix 
determine  the  Markov  chain  process,  considering  they  can  build  the  entire 
network  architecture  [13]. 

The  initial  state  vector  is 

p(t)  =  (Pi . Pn)  with  N  being  the  number  of  states,  and  pi  is  the 

probability  of  starting  in  each  state  of  the  chain. 
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We  get  the  initial  states  from  our  simulation  environment. 


3.  Bandwidth  Transition 

Our  approach  to  establishing  the  bandwidth  transition  is  to  use  the 
transmission  rates  and  their  associate  states.  This  is  a  similar  approach  to  the 
one  taken  in  [8].  For  a  given  bandwidth  level,  the  bandwidth  transmission  model 
is  completely  characterized  by  the  matrices  such  as  P  (transition  probability 
associated  to  first/initial  state)  and  M  (transition  probability  associated  to  the  next 
level). 


4.  Stationary  Distribution 

As  discussed  in  Chapter  III,  in  our  model,  we  assume  that  the  Markov 
chain  is  time-homogeneous  so  that  the  process  can  be  described  by  a  single, 
time-independent  matrix  Pjj.  Then  the  vector  n  is  called  a  stationary  distribution. 

5.  Definition  of  State 

In  our  model  we  define  state  as  the  average  transmit  power  or  average 
pathloss.  The  state  shows  the  level  of  power  and  attenuation  so  that  the 
transition  matrix  consists  of  the  probability  of  decreasing  or  increasing  the  power 
or  attenuation  level. 

6.  INTEGRATION  OF  MARKOV  CHAIN  WITH  RSVP 

RSVP  is  dynamic  in  terms  of  node  status  but  it  is  not  dynamic  in  terms  of 
bandwidth  allocation.  The  Markov  chain  process  is  used  to  make  RSVP  dynamic 
considering  bandwidth  allocation.  RSVP  uses  the  output  vector  of  the  Markov 
chain  process  as  feedback  that  it  can  use  to  decide  which  nodes  need 
bandwidth,  and  thus,  the  bandwidth  can  be  allocated  dynamically. 

The  integration  of  the  Markov  chain  with  the  RSVP  is  illustrated  by  the 
example  shown  in  Figure  6. 
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Figure  6.  Sample  communication  network  diagram  to  illustrate  the  integrated 

Markov  chain  and  RSVP  algorithm. 

In  this  diagram  node  A  wants  to  send  a  message  to  node  G.  The  routing 
algorithm  determines  the  path  (the  blue  line)  and  RSVP  determines  the  required 
bandwidth  that  the  sender  requests  of  the  receiver.  Note  that  the  routing 
algorithm  used  in  this  thesis  is  either  Open  Shortest  Path  First  (OSPF)  or 
Bellman-Ford  [25], [26]. 

The  state  of  a  node  is;  based  on  the  assumption  that  the  channel  quality 
(transmit  power  or  path  loss  in  our  model)  is  between  20  dB  and  50dB.  In  that 
case,  the  difference  between  states  is  3dB.  The  first  state  is  20-23dB,  and  the 
next  state  is  23-26db.  By  adding  3dB  for  each  state,  the  last  state  has  a  channel 
quality  of  47-50dB. 

The  transition  probability  matrix  has  the  probabilities  of  switching  states. 
The  first  step  is  to  modify  the  probability  transition  matrix  by  computing  the 
cumulative  sum  for  each  row  since  this  will  determine  the  transition  between 
states. 
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At  time  t=0,  we  have  the  initial  and  transition  matrices  and  the  initial 
probability  vector.  The  Markov  chain  process  uses  the  random  walk  procedure 
which  is  a  mathematical  formulization  of  a  path  that  consists  of  a  succession  of 
random  steps  that  are  discrete  and  of  fixed  length. 

The  second  step  is  choosing  a  random  value,  which  is  then  compared  with 
the  modified  transition  matrix. 

•  If  the  random  value  is  lower  than  the  current  value,  the  state  is 
decreased  by  one. 

•  If  the  random  value  is  higher  than  the  current  value  but  lower  than 
the  next  value,  the  link  between  the  nodes  remains  the  same. 

•  If  it  is  higher  than  the  next  value,  the  state  increases  by  one. 

For  instance,  assuming  that  the  state  of  the  link  between  node  A  to  node 
B  in  Figure  6  is  4.  The  probability  of  changing  state  is  0.4.  The  state  remains  the 
same  if  the  random  value  is  0.3.  If  the  random  value  is  0.5,  the  state  of  the  link 
increases  by  one  and  becomes  5. 

If  the  transition  matrix  is  stationary,  the  state  will  converge  to  a  number  in 
the  Markov  Chain  process.  This  allows  us  to  determine  the  future  state  of  that 
link.  As  an  example,  at  time  t=0,  the  minimum  state  level  is  3.  At  time  t=n,  the 
minimum  state  level  would  converge  to  state  6.  By  taking  into  consideration  this 
change,  the  allocated  bandwidth  between  sender  and  receiver  can  be  increased 
or  decreased. 

The  output  vector  of  this  process  shows  us  how  this  wireless  network  will 
be  shaped.  After  the  process,  we  will  deduce  and  give  feedback  to  RSVP  about 
whether  to  increase  or  decrease  the  bandwidth  or  remain  the  same. 

The  steps  for  the  Markov  chain  algorithm  are  as  follows; 

1 .  Define  the  transition  matrix  and  the  initial  vector. 

2.  Define  the  output  vector 

3.  Define  the  state  of  the  Markov  chain  process 

4.  Obtain  a  random  value  between  0  and  1 . 
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5.  Take  the  first  value  of  the  initial  vector  and  use  this  random  value 
and  the  initial  value  and  compare  it  to  the  transition  matrix  value. 

a)  Check  whether  this  is  the  first  or  last  state  (it  behaves 
differently  in  different  cases). 

b)  If  the  random  value  is  lower  than  the  transition  matrix  value, 
decrease  the  state  by  one. 

c)  If  it  is  equal  or  higher  than  current  value  but  lower  than  next 
value,  the  state  stays  the  same. 

d)  If  it  is  higher  than  the  next  value,  the  state  increases  by  one. 

6.  Find  the  output  vector. 

7.  If  it  is  stationary,  find  when  and  to  which  number  output  vector  will 
converge. 

Step  5  is  repeated  as  many  times  as  necessary  until  the  output  vector 

ends. 

After  the  Markov  chain  calculates  the  state  of  the  channel,  feedback  can 
be  given  to  RSVP  using  the  output  vector.  In  order  to  allocate  bandwidth 
dynamically,  this  thesis  proposes  two  methods: 

•  Use  refresh  messages  which  are  sent  periodically,  so  that  reserved 
bandwidth  can  be  decreased  or  increased. 

•  Reduce  bandwidth  using  the  model  proposed  in  [24], 
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V.  PERFORMANCE  AND  EVALUATION 


In  this  chapter,  we  presented  the  results  of  a  simulation  study,  conducted 
in  Qualnet,  a  wireless  communication  simulation  platform.  We  use  the 
simulations  to  demonstrate  and  validate  the  Markov  chain  and  RSVP  integrated 
algorithm  described  in  Chapter  IV.  As  a  benchmark,  we  use  the  traditional  RSVP 
algorithm  to  compare  against  the  integrated  Markov  Chain  and  RSVP  algorithm 
presented  in  this  thesis.  The  comparative  analysis  will  allow  us  to  understand 
whether  the  integrated  approach  provides  significantly  better  results  than  the 
traditional  RSVP. 

A.  SIMULATION  DESCRIPTION 

We  simulate  four  different  simulation  networks  that  consist  of  5,  10,  15 
and  30  nodes,  respectively.  The  scenario  dimension  is  1500  m  X  1500  m.  The 
altitude  is  1500m  above  sea  level.  Different  simulation  times  are  chosen, 
specifically  5  sec,  30  sec,  5  min  and  30  min  since  there  needs  to  be  at  least  5-10 
sec  for  the  Markov  chain  process  to  run  effectively.  The  mobility  model  of  nodes 
is  random  way  point  that  simulates  the  movement  of  the  UGV  nodes.  The  node 
maximum  speed  is  10  meter  per  second  (mps).  The  chosen  routing  protocols  are 
Bellman-Ford  and  OSPF.  The  buffer  size  is  16384  bytes.  The  subnets  are 
designed  using  different  numbers  of  nodes  and  different  distances.  Instead  of 
using  a  real  terrain  environment,  the  default  wireless  terrain  data  is  applied.  The 
exact  lists  of  these  parameters  of  simulation  are  shown  in  Table  1 . 
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Table  1 .  Simulation  parameters  used  in  Qualnet. 


TERRAIN 

NETWORK  LAYER 

Coorinate  System 

Cartesian 

Network  Protocol 

IP 

Terrain  Dimension 

1500,1500 

ROUTING  PROTOCOL 

Altitude 

1500,0 

Routing  Protocol 

Bellman  Ford-OSPF 

Weather  Mobility 

10S 

TRANSPORT  PROTOCOL 

CHANNEL  PROPERTIES 

TCP  Sender/Receiver  Buffer 

16384 

Propagation  Channel 

Frequency 

2.4  GHZ 

BATTERY  MODEL 

Propagation  Model 

Statistical 

Battery  Model 

None 

Prop.  Pathloss  Model 

Two-Way 

MAC  LAYER 

Prop.  Shadowing  Model 

Constant 

Link  Prop.  Delay 

IMS 

Prop.  Shadowing  Mean 

4.0 

Link  Bandwidth 

10MB 

Prop.  Fading  Model 

None 

Link  RX/TX  Frequency 

13.17GHz 

Prop.  Speed 

3e8 

Link  RX/TX  Antenna  Height 

30  M 

Prop.  Limit 

-111.0 

Link  TX  Power 

30 

Prop.  Max.  Distance 

0 

Link  RX  Sensitivity 

-80 

Prop.  Comm.  Proximity 

400 

Propogation  Update  Ratio 

0 

PHYSICAL  LAYER 

PHY  Model 

PHY  802.11b. 

PHY  802.11b  Data  Rate 

1,2,6,11  MB 

PHY  802.1 1b  Tx  Power 

15 

PHY  802.11b  Rx  Power  (for 

each  different  data  rate) 

-94,  -91,-87,  -83 

Antenna  Model 

Omnidirectional 

Antenna  Gain 

0 

Antenna  Height 

1.5 

Antenna  Efficiency 

0.8 

Antenna  Mismatch  Loss 

0.3 

Antenna  Cable  Loss 

0 

Antenna  Conncetion  Loss 

0.2 

PHY  Temperature 

290  K 

PHY  Noise  Factor 

10 

34 


An  example  wireless  network  simulated  in  Quainet  with  15  nodes  and  their 
respective  subnets  is  shown  in  Figure  7. 
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Figure  7.  An  example  wireless  communication  network  consist  of  1 5  nodes  that 
was  simulated  in  Quainet.  The  dashed  lines  represent  the  wireless 
connectivity  between  nodes  within  a  subnet. 


B.  SIMULATION  RESULTS  AND  ANALYSIS 

The  performance  metrics  used  for  analysis  are  average  signal  power, 
average  pathloss  and  node  utilization.  These  parameters  are  as  follows: 

Average  Signal  Power:  Average  signal  power  is  measured  in  dBm  from 
the  beginning  of  simulation  up  to  the  specified  time  in  the  Timestamp  column. 
These  values  are  chosen  to  be  5  sec,  30  sec,  5  min  and  30  min  in  our 
simulations. 

Average  Pathloss:  Average  pathloss  (in  dB)  is  the  reduction  in  the  power 
density  (attenuation)  of  the  signal.  In  the  simulations  it  is  a  measure  from  the 
physical  layer  at  the  beginning  of  the  simulation  up  to  the  line  specified  in  the 
Timestamp  column. 
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Node  Utilization:  Average  utilization  of  the  nodes  in  terms  of  power 
measured  from  physical  layer  from  the  beginning  of  the  simulation  up  to  the  time 
specified  in  the  Timestamp  column. 

A  total  of  34  simulations  were  executed.  Due  to  space  constraints,  we  are 
showing  a  subset  of  the  simulations.  The  results  shown  depict  the  main 
simulations  that  reflect  the  best  results  that  will  allow  comparison  and  analysis. 

1.  5-Node  Wireless  Communication  Network 
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Figure  8.  The  5-node  wireless  network  simulated  in  the  Qualnet  environment. 


The  simulation  time  is  30  min  and  the  chosen  data  rate  is  2  mbps  for  the 
5-node  network,  as  shown  in  Figure  8.  The  average  power/pathloss  results  of 
this  network  are  shown  in  Table  2. 
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Table  2.  5-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-76 

3 

89 

8 

2 

-80 

2 

95 

9 

3 

-75 

3 

91 

9 

4 

9 

10 

42 

4 

5 

6 

10 

62 

6 

The  average  power  is  between  -91  dB  to  12  dB.  We  define  the  state  level 
as  10dB  and  the  state  of  -99  to  -90  dB  is  1  and  the  state  of  -89  to  80  dB  is  2  and 
so  on.  This  is  similar  to  when  we  consider  average  pathloss.  Looking  at  the 
results,  we  can  see  that  the  change  of  average  pathloss  on  node  4  and  5  is  quite 
large  (i.e.,  9  to  42  and  6  to  62,  respectively).  Although  the  chosen  path  can  be 
seen  by  looking  at  the  table  because  there  are  merely  5  nodes,  it  is  difficult  to 
designate  the  path  for  large  networks.  Since  in  the  Markov  chain  process,  the 
transition  probability  matrix  consists  of  the  probability  of  switching  state,  it  helps 
us  to  determine  which  state  is  going  to  change.  Here  the  path  is 

Path:  1-4-5 

After  the  Markov  chain  process  is  executed,  we  get  the  results  shown  in 
Table  3. 

The  simulation  time  is  30  min  and  the  data  rate  increased  to  11  mbps  for 
those  nodes  on  the  path.  The  comparative  results  are  shown  in  Table  3. 


Table  3.  Results  before  and  after  Markov  chain  process  for  a  5  node  network 
with  the  data  path  1-4-5.  Node  utilization  and  BER  results  after  the 
Markov  chain  (AMC)  and  before  the  Markov  chain  (BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

1 

0.000023 

330/2454 

0.000039 

299/2556 

4 

0.0033 

325/2563 

0.00057 

141/2546 

5 

0.00513 

280/2264 

0.00623 

225/2332 
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The  node  utilization  and  the  bit  error  rate  (BER)  at  node  4  and  5  increase 
slightly.  With  these  results  it  is  difficult  to  gauge  whether  the  integration  of  the 
Markov  chain  with  integrated  RSVP  works  better  or  worse  than  RSVP  alone.  This 
is  specifically  due  to  the  small  network  size. 


2.  10-Node  Wireless  Communication  Network 

The  next  simulation  network  studied  is  a  10-node  network  shown  in  Figure 
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Figure  9.  The  1 0-node  wireless  network  simulated  in  the  Qualnet  environment. 


The  simulation  time  is  30  min,  and  the  chosen  data  rate  is  2  mbps  for  the 
10-node  network.  The  average  power/pathloss  results  are  shown  in  Table  4. 
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Table  4.  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-76 

3 

90 

9 

2 

-74 

3 

94 

9 

3 

-58 

5 

91 

9 

4 

-72 

3 

90 

9 

5 

-81 

2 

94 

9 

6 

7 

10 

56 

5 

7 

-77 

3 

91 

9 

8 

7 

10 

56 

5 

9 

-76 

3 

92 

9 

10 

-75 

3 

90 

9 

The  average  power  is  between  -87  dB  to  7  dB.  We  define  the  state  level 
as  10dB.  The  state  of  -99  to  -90  dB  is  1  and  the  state  of  -89  to  80  dB  is  2  and  so 
on.  We  use  a  similar  approach  for  average  pathloss  value.  The  Markov  chain 
process  takes  into  consideration  the  average  loss  and  calculates  that  node  6  and 
node  8  will  transition  to  a  new  state.  The  path  in  which  the  states  are  changed  is 
chosen  as 

Path:  2-6-8-10 

The  simulation  time  is  30  min,  and  the  data  rate  increased  to  1 1  mbps  for 
those  nodes  on  the  path.  The  comparative  results  are  shown  in  Table  5. 
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Table  5.  Results  before  and  after  Markov  chain  process  for  a  10  node 
network  with  the  data  path  2-6-8-10.  Node  utilization  and  BER 
results  after  the  Markov  chain  (AMC)  and  before  the  Markov  chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

2 

0.0002 

879/2892 

0.0003 

954/3489 

6 

0.0003 

748/2975 

0.0004 

682/3241 

8 

0.0003 

775/3164 

0.0003 

685/3315 

10 

0.0026 

725/3263 

0.0022 

734/3474 

After  we  modify  the  simulation,  the  node  utilization  does  not  change  but 
BER  decreases.  It  can  be  concluded  here  that  in  terms  of  BER,  the  integration  of 
Markov  Chain  process  with  RSVP  works  better. 


15  Node  Wireless  Communication  Network 
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Figure  1 0.  The  1 5  node  wireless  network  simulated  in  the  Qualnet  environment. 


The  next  network  studied  is  a  15-node  network,  as  shown  in  Figure  10. 
The  simulation  time  is  30  min  and  the  chosen  data  rate  is  2mbps  for  the  15-node 

network.  The  average  power/pathloss  results  are  shown  in  Table  6. 
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Table  6.  15  node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

- 

- 

- 

- 

2 

-72 

3 

91 

9 

3 

3 

10 

69 

6 

4 

-75 

3 

90 

9 

5 

3 

10 

71 

7 

6 

-78 

3 

92 

9 

7 

-80 

2 

93 

9 

8 

4.8 

10 

63 

6 

9 

3.9 

10 

68 

6 

10 

-77 

3 

91 

9 

11 

-77 

3 

92 

9 

12 

-74 

3 

92 

9 

13 

3.6 

10 

70 

7 

14 

-77 

3 

92 

9 

15 

-75 

3 

91 

9 

The  state  level  is  10  dB.  The  Markov  Chain  process  takes  into 
consideration  the  average  loss  and  calculates  that  node  6  and  node  8  is  going  to 
have  new  state.  The  path  in  which  nodes  level  are  changed  is  chosen  as 

Path:  3-8-9-15 

The  simulation  time  is  30  min  and  the  data  rate  is  increased  to  11  mbps 
for  those  nodes  on  the  path.  The  comparative  results  are  shown  in  Table  7. 
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Table  7.  Results  before  and  after  Markov  chain  process  for  a  10  node 
network  with  the  data  path  3-8-9-15.  Node  utilization  and  BER 
results  after  the  Markov  chain  (AMC)  and  before  the  Markov  chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

3 

0.003353 

2657/9824 

0.001044 

2530/8990 

8 

0.002838 

2056/10026 

0.002012 

1385/9130 

9 

0.002442 

1677/9949 

0.001303 

1641/9189 

15 

0.003141 

2295/10297 

0.003141 

2126/9436 

4.  30  Node  Wireless  Communication  Network 
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Figure  1 1 .  Tnode  wireless  network  simulated  in  the  Qualnet  environment. 


The  average  power/pathloss  results  are  shown  in  Table  8. 
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Table  8.  30  node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR.  PATHLOSS 

STATE 

1 

-76 

3 

90 

9 

2 

-75 

3 

90 

9 

3 

-77 

3 

91 

9 

4 

-76 

3 

90 

9 

5 

-78 

3 

90 

9 

6 

-68 

4 

88 

8 

7 

-77 

3 

91 

9 

8 

3 

10 

71 

7 

9 

-75 

3 

90 

9 

10 

-75 

3 

90 

9 

11 

-70 

3 

91 

9 

12 

-76 

3 

91 

9 

13 

-76 

3 

90 

9 

14 

-74 

3 

92 

9 

15 

2.3 

10 

70 

7 

16 

-78 

3 

91 

9 

17 

-78 

3 

91 

9 

18 

-75 

3 

89 

8 

19 

-64 

4 

89 

8 

20 

-76 

7 

91 

9 

21 

2.8 

10 

70 

7 

22 

2.5 

10 

72 

7 

23 

-75 

3 

91 

9 

24 

-79 

3 

91 

9 

25 

-77 

3 

90 

9 

26 

-76 

3 

91 

9 

27 

-77 

3 

90 

9 

28 

-69 

3 

90 

9 

29 

-81 

2 

92 

9 

30 

-78 

3 

91 

9 

After  the  Markov  Chain  process  runs,  the  path  chosen  to  analyze  is 
Path:  2-8-21-18 

The  simulation  time  is  30  min  and  the  data  rate  increased  to  1 1  mbps  for 
those  nodes  on  the  path.  The  comparative  results  are  shown  in  Table  9. 
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Table  9.  Results  before  and  after  Markov  chain  process  for  a  30-node 
network  with  the  data  path  2-8-21-18.  Node  utilization  and  BER 
results  after  the  Markov  chain  (AMC)  and  before  the  Markov  chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

2 

0.005366 

8194/28445 

0.002870 

9150/27274 

8 

0.004020 

4323/27684 

0.002098 

4916/26605 

21 

0.004214 

5232/27818 

0.002321 

4711/26686 

18 

0.003141 

5828/28377 

0.002446 

6505/27223 

When  we  compare  the  5-node  network  and  the  30-node  network  and 
when  we  compare  the  5  sec  to  30  min  simulation  results,  it  can  be  said  that  the 
simulations  results  make  more  sense  as  the  number  of  nodes  and  simulation 
time  increases.  This  is  particularly  true  for  the,  5-second  simulation  time,  which 
results  in  more  nuanced  results  than  for  other  simulations  (30  sec,  5  min  and  30 
min). 

Although  the  approach  and  results  are  scientifically  correct,  it  should  be 
noted  that  there  are  some  limitations  due  to  the  Qualnet  simulation  program.  The 
most  important  limitation  is  that  Qualnet  supports  only  four  data  rates  (2,4,8,16 
mbps)  in  the  802.1 1  b  wireless  simulation  environment.  This  limitation  restricts  the 
transition  of  bandwidth  effectively. 

Changing  routing  algorithms  does  not  affect  the  results  much  when  the 
number  of  nodes  is  low.  For  instance  the  5  node  network’s  average 
power/pathloss  and  utilization/BER  results  are  shown  for  Bellman  Ford  and 
OSPF  [26], [27]  in  Tables  10-13. 
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Table  10.  Power/path  loss  results  using  the  Bellman  Ford  routing  algorithm  for 

the  5  node  network  where  the  simulation  time  is  30  sec. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-91 

1 

0 

- 

2 

-88 

2 

97 

9 

3 

-91 

1 

0 

- 

4 

10 

11 

12 

1 

5 

12 

11 

0 

- 

Table  1 1 .  Power/path  loss  results  using  the  OSPF  routing  algorithm  for  the  5 
node  network  where  the  simulation  time  is  30  sec. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-90 

1 

0 

- 

2 

-89 

2 

97 

9 

3 

-91 

1 

0 

- 

4 

10 

11 

7 

1 

5 

12 

11 

0 

- 

Table  12.  Node  utilization  and  BER  results  for  using  Bellman  Ford  routing 

algorithm  for  the  5  node  network  where  the  simulation  time  is  30  sec. 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER 

(AMC) 

1 

0.000022 

6/42 

0.000019 

7/48 

4 

0.0035 

8/42 

0.0035 

5/49 

5 

0.0023 

3/43 

0.0023 

3/46 
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Table  13.  Node  utilization  and  BER  results  for  using  the  OSPF  routing 

algorithm  for  the  5  node  network  where  the  simulation  time  is  30  sec. 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER 

(AMC) 

1 

0.000015 

6/42 

0.000021 

7/49 

4 

0.00033 

5/42 

0.0036 

4/48 

5 

0.000015 

1/42 

0.0019 

4/48 

The  30-node  power/pathloss  and  node  utilization  and  BER  results  are 
shown  in  Tables  14-17  for  Bellman  Ford  and  OSPF.  It  can  be  seen  that  the 
differences  are  much  bigger  than  for  5-node  network. 
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Table  14.  Power/path  loss  results  using  the  Bellman  Ford  routing  algorithm  for 

the  30-node  network  where  the  simulation  time  is  30  sec. 


When  we  compare  the  result  of  average  power  and  pathloss  between  the 
Bellman  Ford  and  OSPF,  the  states  of  many  nodes  are  different.  For  instance 
node  21  and  node  22  have  difference  states  considering  pathloss.  The  node  21 
average  pathloss  for  Bellman  Ford  is  64dB,  so  the  state  is  6,  and  it  is  74dB  for 
OSPF,  so  the  state  is  7.  Node  22  is  also  similar. 


Table  16.  Node  utilization  and  BER  results  for  using  Bellman  Ford  routing 
algorithm  for  the  30-node  network  where  the  simulation  time  is  30 

sec. 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER 

(AMC) 

2 

0.0002 

91/563 

0.005 

62/596 

8 

0.0003 

216/585 

0.009 

143/624 

21 

0.003 

216/589 

0.008 

135/624 

18 

0.0001 

48/594 

0.003 

44/628 

Table  17.  Node  utilization  and  BER  results  for  using  the  OSPF  routing 

algorithm  for  the  30-node  network  where  the  simulation  time  is  30 

sec. 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER 

(AMC) 

2 

0.009 

411/3174 

0.04 

341/3111 

8 

0.01 

1073/3298 

0.05 

71 5/3372 

21 

0.01 

949/3325 

0.04 

744/3120 

18 

0.003 

204/3260 

0.001 

1 1 4/3298 
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VI.  CONCLUSION  AND  FUTURE  WORK 


A.  CONCLUSION 

Communication  networks  have  become  an  integral  part  of  life  today. 
However,  one  of  the  main  bottlenecks  of  communication  networks  is  the 
availability  of  bandwidth.  This  is  especially  true  in  the  wireless  environment. 
Another  important  aspect  of  wireless  communication  is  efficiency.  Efficient 
network  resource  management  and  quality  of  service  (QoS)  are  parameters  that 
need  to  be  achieved  especially  when  considering  network  delays.  The 
cooperative  nature  of  UGV  networks  require  that  bandwidth  allocation  be  shared 
fairly  between  individual  UGV  nodes,  depending  on  necessity. 

In  this  thesis,  we  study  the  problem  of  dynamic  bandwidth  provisioning  in 
an  UGV  network.  Specifically,  we  integrate  the  use  of  a  basic  statistical  model, 
known  as  the  Markov  chain  with  a  widely  known,  network  bandwidth  reservation 
protocol,  known  as  the  Resource  Reservation  Protocol  (RSVP).  The  Markov 
chain  results  are  used  with  RSVP  to  identify  specific  bandwidth  allocation 
requirements  along  a  path  such  that  data  transmission  along  that  path  is 
successful.  We  analyze  the  bandwidth  efficiency  and  show  that  the  integration  of 
the  Markov  chain  with  RSVP  provides  higher  bandwidth  guarantees  and  better 
overall  QoS  when  compared  with  using  RSVP  alone  in  wireless  communication 
networks. 

In  our  performance  evaluation,  we  simulated  four  different  network 
environments  that  have  the  same  parameters.  The  algorithm  is  shown  to  have 
better  node  utilization  and  BER  when  the  Markov  chain  predicts  future  channel 
conditions  and  determines  what  bandwidth  requirements  are  needed.  The  results 
from  the  Markov  chain  process  are  then  used  with  RSVP  to  reserve  the 
appropriate  bandwidth  along  individual  paths.  We  have  demonstrated  that  RSVP 
works  better  when  the  Markov  chain  is  integrated  with  it  as  compared  to  when 
only  RSVP  is  used  for  bandwidth  guarantees. 
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B.  FUTURE  WORK 


In  this  thesis,  we  integrate  the  Markov  chain  process  with  RSVP.  RSVP  is 
built  into  the  Qualnet  simulator  and  the  Markov  chain  program  is  written 
separately  and  then  they  are  integrated  manually.  A  potential  future  work  can  be 
to  integrate  them  in  the  Qualnet  environment.  The  Qualnet  simulator  allows  this 
approach.  The  dynamic  Application  Programming  Interface  (API)  in  the  Qualnet 
environment  allows  users  and  programs  to  modify  and  monitor  a  Qualnet 
simulation  dynamically.  Using  the  dynamic  API,  a  user  or  program  can  change 
values  of  variables  and  can  be  notified  when  statistics  change  during  execution. 
The  dynamic  API  operates  between  the  external  interface  API  (from  the 
perspective  of  this  thesis,  this  can  be  the  Markov  chain  process)  and  the 
simulation.  The  external  interface  API  is  responsible  for  communicating  with  the 
external  program  and  sending  commands  from  the  external  program  to  the 
dynamic  API.  The  dynamic  API  interacts  with  the  Qualnet  simulation,  giving  the 
results  of  commands  back  to  the  external  interface,  API,  in  turn,  sends  results 
back  to  the  external  program  [28].  Due  to  time  considerations,  this  method  was 
not  pursued  in  this  thesis. 

Another  reason  that  the  Markov  chain  process  with  RSVP  should  be 
integrated  into  the  Qualnet  environment  is  that  the  work  that  has  been  done  in 
this  thesis  uses  the  values  of  the  Markov  Chain  and  RSVP,  and  then  those 
values  are  changed  accordingly.  So,  in  some  sense  it  is  not  necessarily  dynamic. 
A  more  pure  dynamic  approach  in  which  the  system  actually  changes  with  each 
time  t  when  a  new  bandwidth  allocation  is  requested  should  be  studied.  In  this 
thesis  we  can  only  look  at  small  node  networks  because  we  are  studying  and 
obtaining  the  measurements  first  before  making  the  bandwidth  allocation.  An 
integrated  approach  using  Qualnet,  may  help  in  network  scale  as  well  (looking  at 
larger  networks  of  100  nodes). 

The  modeled  scenario  is  limited  in  scope,  and  future  work  can  be  spent 
working  on  more  expanded  scenarios.  Although  it  can  be  more  difficult  to 
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analyze,  the  dimension  of  the  network  area  or  the  number  of  nodes  can  be 
increased  to  get  more  real  time  values. 

It  needs  to  be  noted  that  in  future  work,  another  simulator  such  as  OPNET 
can  be  used.  This  can  avoid  some  of  the  Qualnet  limitations  such  as  data  rate 
types. 
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APPENDIX  A.  SUPPLEMENTAL  SIMULATION  RESULTS 


This  appendix  provides  the  supplemental  simulations  results  that  were 
analyzed  but  due  to  space  not  included  in  Chapter  5. 


A.  SIMULATION  RESULTS  FOR  5  NODE  WIRELESS  COMMUNICATION 
NETWORK 

The  average  power  is  between  -91  dB  to  12  dB.  We  define  the  state  level 
as  10dB  and  The  state  of  -99  to  -90  dB  is  1  and  the  state  of  -89  to  80  dB  is  2  and 
so  on.  This  is  similar  when  we  consider  average  pathloss. 


1.  Simulation  Time  :  5  sec,  Data  Rate  :  2mbps 


Table  18.  5-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-91 

1 

0 

- 

2 

-91 

1 

0 

- 

3 

-91 

1 

0 

- 

4 

10 

11 

0 

- 

5 

12 

11 

0 

- 

2.  Simulation  Time  :  30  sec,  Data  Rate  :  2mbps 


Table  19.  5-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-91 

1 

0 

- 

2 

-88 

2 

97 

9 

3 

-91 

1 

0 

- 

4 

10 

11 

12 

1 

5 

12 

11 

0 

- 
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3. 


Simulation  Time  :  5  min,  Data  Rate  :  2  mbps 


Table  20.  5-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-88 

1 

96 

9 

2 

-75 

3 

92 

9 

3 

-72 

3 

91 

9 

4 

10 

11 

17 

1 

5 

7 

10 

25 

2 

4.  Simulation  Time  :  30  min,  Data  Rate  :  2  mbps 


Table  21 .  5-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-76 

3 

89 

8 

2 

-80 

2 

95 

9 

3 

-75 

3 

91 

9 

4 

9 

10 

42 

4 

5 

6 

10 

62 

6 

THE  PATH:  1-4-5 

5.  Simulation  Time  :  30  min,  Data  Rate  :  11  mbps 


Table  22.  Results  before  and  after  Markov  Chain  process  for  a  5  node  network 
with  the  data  path  1-4-5.  Node  utilization  and  BER  results  after  the 
Markov  Chain  (AMC)  and  before  the  Markov  Chain  (BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER 

(AMC) 

1 

0.000023 

330/2454 

0.000039 

299/2556 

4 

0.0033 

325/2563 

0.00057 

141/2546 

5 

0.00513 

280/2264 

0.00623 

225/2332 
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B.  SIMULATION  RESULTS  FOR  10-NODE  WIRELESS  COMMUNICATION 
NETWORK 

1.  Simulation  Time  :  5  sec,  Data  Rate  :  2mbps 


Table  23.  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-81 

2 

90 

9 

2 

-83 

2 

94 

9 

3 

-83 

2 

95 

9 

4 

-83 

2 

94 

9 

5 

-87 

2 

98 

9 

6 

3 

10 

68 

6 

7 

-84 

2 

96 

9 

8 

3 

10 

74 

7 

9 

-84 

2 

96 

9 

10 

-86 

2 

97 

9 

2.  Simulation  Time  :  30  sec,  Data  Rate  :  2mbps 


Table  24.  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-82 

2 

91 

9 

2 

-82 

2 

93 

9 

3 

-83 

2 

95 

9 

4 

-82 

2 

94 

9 

5 

-86 

2 

97 

9 

6 

3 

10 

71 

7 

7 

-83 

2 

95 

9 

8 

4 

10 

68 

6 

9 

-84 

2 

96 

9 

10 

-87 

2 

98 

9 
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3. 


Simulation  Time  :  5  min,  Data  Rate  :  2mbps 


Table  25.  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-85 

2 

93 

9 

2 

-69 

4 

93 

9 

3 

-78 

3 

93 

9 

4 

-63 

4 

93 

9 

5 

-84 

2 

95 

9 

6 

6 

10 

63 

6 

7 

-76 

3 

91 

9 

8 

6 

10 

60 

6 

9 

-77 

3 

92 

9 

10 

-80 

2 

94 

9 

4.  Simulation  Time  :  30  min,  Data  Rate  :  2mbps 


Table  26.  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

-76 

3 

90 

9 

2 

-74 

3 

94 

9 

3 

-58 

5 

91 

9 

4 

-72 

3 

90 

9 

5 

-81 

2 

94 

9 

6 

7 

10 

56 

5 

7 

-77 

3 

91 

9 

8 

7 

10 

56 

5 

9 

-76 

3 

92 

9 

10 

-75 

3 

90 

9 

58 


Path:  2-6-8-10 


Table  27.  Results  before  and  after  Markov  Chain  process  for  a  10  node 
network  with  the  data  path  2-6-8-10.  Node  utilization  and  BER 
results  after  the  Markov  Chain  (AMC)  and  before  the  Markov  Chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

2 

0.0002 

879/2892 

0.0003 

954/3489 

6 

0.0003 

748/2975 

0.0004 

682/3241 

8 

0.0003 

775/3164 

0.0003 

685/3315 

10 

0.0026 

725/3263 

0.002265 

734/3474 

C.  SIMULATION  RESULTS  FOR  15  NODE  WIRELESS  COMMUNICATION 
NETWORK 

1.  Simulation  Time  :  5  sec,  Data  Rate  :  2mbps 


Table  28.  15-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

- 

- 

- 

- 

2 

-84 

2 

94 

9 

3 

1.7 

10 

75 

7 

4 

-81 

2 

93 

9 

5 

2.2 

10 

78 

7 

6 

-85 

2 

95 

9 

7 

-84 

2 

94 

9 

8 

2.6 

10 

63 

6 

9 

3 

10 

77 

7 

10 

-86 

2 

98 

9 

11 

-88 

2 

97 

9 

12 

-87 

2 

97 

9 

13 

1 

10 

86 

8 

14 

-85 

2 

97 

9 

15 

-86 

2 

96 

9 
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4. 


Simulation  Time  :  30  min,  Data  Rate  :  2mbps 


Table  31 .  10-node  network  average  power/pathloss  results. 


NODE 

AVR.  POWER 

STATE 

AVR. 

PATHLOSS 

STATE 

1 

- 

- 

- 

- 

2 

-72 

3 

91 

9 

3 

3 

10 

69 

6 

4 

-75 

3 

90 

9 

5 

3 

10 

71 

7 

6 

-78 

3 

92 

9 

7 

-80 

2 

93 

9 

8 

4.8 

10 

63 

6 

9 

3.9 

10 

68 

6 

10 

-77 

3 

91 

9 

11 

-77 

3 

92 

9 

12 

-74 

3 

92 

9 

13 

3.6 

10 

70 

7 

14 

-77 

3 

92 

9 

15 

-75 

3 

91 

9 

Path  :  3-8-9-15 


Table  32.  Results  before  and  after  Markov  Chain  process  for  a  10  node 
network  with  the  data  path  3-8-9-15.  Node  utilization  and  BER 
results  after  the  Markov  Chain  (AMC)  and  before  the  Markov  Chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

3 

0.003353 

2657/9824 

0.001044 

2530/8990 

8 

0.002838 

2056/10026 

0.002012 

1385/9130 

9 

0.002442 

1 677/9949 

0.001303 

1641/9189 

15 

0.003141 

2295/10297 

0.003141 

2126/9436 
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SIMULATION  RESULTS  FOR  30  NODE  WIRELESS  COMMUNICATION 
NETWORK 

1.  Simulation  Time  :  5  sec,  Data  Rate  :  2mbps 


Table  33.  30-node  network  average  power/pathloss  results. 


NODE  AVR.  POWER  STATE  AVR.  STATE 

PATHLOSS 


Path:  2-8-21-18 


Table  37.  Results  before  and  after  Markov  Chain  process  for  a  10  node 
network  with  the  data  path  2-8-21-18.  Node  utilization  and  BER 
results  after  the  Markov  Chain  (AMC)  and  before  the  Markov  Chain 

(BMC). 


NODE 

UTILIZATION 

(BMC) 

BER  (BMC) 

UTILIZATION 

(AMC) 

BER  (AMC) 

2 

0.005366 

8194/28445 

0.002870 

9150/27274 

8 

0.004020 

4323/27684 

0.002098 

4916/26605 

21 

0.004214 

5232/27818 

0.002321 

4711/26686 

18 

0.003141 

5828/28377 

0.002446 

6505/27223 
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APPENDIX  B.  MARKOV  CHAIN  CODE  IN  QUALNET 

ENVIRONMENT 


This  appendix  provides  the  code  for  the  Markov  Chain,  which  is  written  in 
C  ++  in  the  Qualnet  environment. 


/kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

kkkkkkkkk 

k 

*mchain : Finite  State  Markov  Chain  Process 

*main  (command  line  parsing  and  database  implementation) 

k 

*purpose:  To  estimate  next  states  of  channel 

*author:  Yavuz  SAGIR 

*Date  First  Writen  :  5  April  2013 

*Date  Last  Modified  :  10  May  2013 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 
kkkkkkkk j 


#include 

#include 

#include 

#include 

#include 

#include 

#include 


<iostream> 

<f stream> 

<cstring> 

<cstdlib> 

<iomanip> 

<string> 

<time . h> 


using  namespace  std; 

//const  int  MAXSTATE  =  15; 
const  int  lengthChain  =  15; 


/ /parameters 

int  channel [ 15 ]; //output  vector  is 

double  action;  //randomly  generated  number  with  which  compare  it  to 
double  PI  [15]  [15] ; 

//  read  TransMatrix  function  reads  the  initial  transition  matrix  from 
the  file  before  executing  the  Markov  Chain  process 


void  readTransMatrix  (const  char*  trans_matrix) 

{ 

FILE*  file  =  fopen  (trans_matrix,  "r"); 
char  i  [80] ; 
double  d  =0; 
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} 


fscanf  (file,  "%s",  &i) ; 

while  (!feof  (file)) 

{ 

d  =  at of  (i) ; 
printf  ("%f  ",  d) ; 
fscanf  (file,  "%s", 

} 

fclose  (file) ; 


&i)  ; 


//printlnitial  function  prints  the  initial  transition  matrix 
void  printlnitial  ( ) 

{ 

printf ("the  initial  values  of  PI  :\n"); 
for  (int  i=0; i<=14; i++) 

{ 

double  temp  =  0; 

for  (int  k=0; k<=14; k++) 

{ 

Pl[i][k]  =  (rand() %101) /100 . 0; 
printf ("%f-", PI  [i]  [k] ) ; 

} 

printf ( " \n" ) ; 

} 

} 

//printOutputVector  shows  the  output  vector  after  the  Markov  chain 
process  is  executed, 
void  printOutputVector ( ) 

{ 

for  (int  i=0; i<=14; i++) 

{ 

printf ("%d  -  ",  channel  [i] ) ; 

} 


int  main (int  argc,  char  *argv[] ) 

{ 

/*  initialize  random  seed:  */ 
srand  (time (NULL) ) ; 

//printing  the  initial  matrix 
printlnitial  () ; 
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//for  (int  i=0; i<=lengthChain; i++)  //check  every  row 

//{ 

int  i  =  channel  [0];  //assign  channel's  first  member  to  row 

for  (int  t=l ;  t<=15 ;  t  +  +  )  //discrete  time  steps  such  as  t=0,  t=l . t=n 

{ 

for  (int  j  =  1 ; j<=lengthChain; j++)  //check  every  column 

{ 

action  =  rand ( )  %  101; 

action  =  action/100.0;  //generate  a  number  between  0  and  1 
if  (  (i  ==  0)& (action  <P1 [ i ][ j ])) //previous  sample  is  just  first 

value  and  compare  random  number  and  transition  matrix 

{ 

channel  [j]  =  1; 

printf("l  is  assigned  to  channel,  no  switch\n"); 

} 

else  if  ( (i  ==  0) & (action>Pl  [i]  [  j  +  1] ) ) 

{ 

channel  [j]  =  3; 

printf("3  is  assigned  to  channel,  increase\n" ) ; 

} 

else 

{ 

channel  [j]  =  2; 

printf("2  is  assigned  to  channel\n")  ; 

} 

if  ( (i  ==  lengthChain-1 )& (action  <Pl[i][j])) 

{ 

channel  [j]  =  1 ; 

printf("l  is  assigned  to  channel,  last  to  first\n"); 

} 

else  if  ( (i  ==  lengthChain-1 )& (action  >P1  [ i ]  [ j  +  1 ] ) ) 

{ 

channel  [j]  =  lengthChain; 

printf("last  value  of  chain  is  assigned  to  channel\n" ) ; 

} 

else 

{ 

channel  [j]  =  lengthChain-1 ; 

printf("the  last  value  is  checked,  no  change\n"); 

} 

if  (action  <Pl[i] [j])  //compare  random  value  and  the  transition 
matrix 

{ 

channel  [j]  =  i-1; 

printf("when  the  value  of  j  is  :  %d,  decrease  one\n",j); 

} 

else  if  (action  >  P 1  [ i ]  [j]) 

{ 

channel  [j]  =  i+1; 

printf("when  the  value  of  j  is  :  %d,  increase  one\n",j); 

} 
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else 

{ 

channel  [  j ]  =  i ; 

printf("when  the  value  of  j  is  :  %d,  no  switch\n" , j ) ; 

} 

} 

printf("time  step  :  %d\n",t); 

for  (int  i=0; i<=14; i++) 

{ 

printf("%d  -  ",  channel  [i] ) ; 

} 

} 

//} 

//printf  channel  state  output  vector 

printf ( "channel  state  output  vector  :\n"); 
printOutputVector ( )  ; 
system ( "pause" )  ; 

} 
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